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1.  In  this  report,  we  compared  the  FY87  ALERT  model's  BP15  Program  Objective 
Memorandum  (POM)  predictions  and  other  less  sophisticated  forecasting  approaches 
to  actual  FY87  obligations.  ALERT  was  more  accurate  than  the  other  forecasting 
models.  Indeed  the  ALERT  forecasts  for  the  total  BP15  obligations  were  within 
two  percent  of  the  actual.  We  provided  some  suggestions  for  the  continued 
enhancement  of  ALERT.  We  provide  all  of  our  conclusions  and  recommendations  in 
Attachment  1. 

2.  Our  point  of  contact  is  Rob  Lucas  or  Larry  Collins,  HQ  AFLC/MMMAA, 

AUTOVON  787-5249  or  787-5314. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS 

1.  In  order  to  validate  ALERT  and  continually  improve  ALERT,  we  should  annually 
compare  the  ALERT  POM  projections  against  actual  obligations  for  the  first  fiscal  year 
of  that  program  year's  budget. 

2.  A  comparison  of  the  1987  ALERT  POM  forecasts  (developed  in  1984)  to  actual  1987 
obligations  shows  ALERT'S  total  DPI5  spares  forecast  was  remarkably  accurate  (within  2 
percent). 

3.  Although  not  statistically  accurate,  ALERT  outperformed  other  "less  sophisticated" 
forecasting  models  both  by  weapon  system  and  by  total  BPI5  spares  requirement. 

4.  We  need  to  expand  the  historical  data  base  to  include  other  variables  to  forecast  the 
13P15  budget  to  reduce  the  forecast  error  and  to  explain  (in  a  cause  and  effect  manner) 
changes  in  requirements. 

5.  We  need  to  examine  other  13 P 1 5  POM  forecasting  models  (either  in-house  or 
contractor)  in  order  to  guarantee  forecast  model  superiority  and  accuracy. 

6.  We  need  to  analyze  the  accuracy  and  sensitivity  of  the  age  of  the  fleet  and  value  of 
the  fleet  variables  on  the  ALERT  model  forecasts. 

RECOMMENDATIONS 

1.  Continue  to  use  ALERT.  (OPR:  HQ  AFLC/MMM) 

2.  repeat  the  validation  effort  for  FY88  using  ALERT  POM  projections  from  FY85. 
(OPR:  HQ  AFLC/MMM  A) 

3.  Continue  to  refine  the  ALERT  model  by  enhancing  the  input  variables  and  the  data 
base.  (OPR:  HQ  AFLC/MMMA) 

4.  Continue  to  experiment  with  other  13PI5  POM  forecasting  approaches  and  validate 
their  results  against  those  of  ALERT.  (OPR:  HQ  AFLC/MMMA  OCR:  HQ  AFLC/MMMI) 
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ABSTRACT 

This  report  documents  an  in-house  validation  of  the  Air  Logistics  Early  Requirements 
Technique  (ALERT),  developed  in  1983  by  HQ  AFLC/MMMA.  ALERT  has  been  used  since 
fiscal  year  (FY)  1984  to  baseline  the  Aircraft  Replenishment  Spares  Budget  (BPI5) 
Program  Objective  Memorandum  (POM)  submission  to  HQ  USAF.  This  report  compares 
the  ALERT  model's  POM  forecasts  for  1987  and  other  simple  budget  forecasting 
approaches  to  the  actual  FY87  first-year  obligations  for  BP  1 5  spares. 
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EXECUTIVE  SUMMARY 


For  five  years,  Air  Force  Logistics  Command  (AFLC)  has  used  ALERT  as  a  tool  to 
forecast  the  budget  program  15  (BP15),  aircraft  spares  Program  Objective  Memorandum 
(POM)  peacetime  operating  stock  requirement.  ALERT  is  a  regression-based  model  that 
uses  historical  budget  data  to  develop  budget  estimates  by  weapon  system.  These 
individual  forecasts  are  then  aggregated  into  an  overall  BP  1 5  POM  forecast,  for  the  first 
POM  period  past  the  budget  projections  (three  years  beyond  the  current  fiscal  year)  and 
for  the  four  subsequent  POM  years.  The  ALERT  estimation  process  allows  for  a 
management  review  of  the  statistically  forecasted  values. this  reporr^we  compared 
ALERT'S  projections  to  other  forecasting  methods.  )#e  also  compared  ALERT'S 
statistically  derived  results  against  ALERT'S  results  after  the  management  review.  We 
found  ALERT  was  more  accurate  than  other  less  sophisticated  forecasting  approaches. 
Indeed  ALERT'S  forecasts  for  FY87  were  within  2  percent  of  the  actual  obligations  for 
FY87.  Wc  propose^continuing  annual  validation  of  ALERT  forecasts  and  reccmmenckother 
efforts  to  improve  AFLC's  ability  to  forecast  BP15  POM  requirements. 
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CHAPTER  I 


THE  PROBLEM 


PROBLEM  STATEMENT 

The  Air  Logistics  Early  Requirement  Technique  (ALERT)  has  been  accepted  by  the  Air 
Force  Logistics  Command  (AFLC)  and  the  Air  Staff  as  the  long-range  forecasting  tool  to 
forecast  budget  program  15  (BP  15),  aircraft  replenishment  spares  requirements.  Several 
early  studies  to  determine  the  accuracy  of  ALERT  forecasts  were  inconclusive,  primarily 
due  to  a  dearth  of  data  and  an  inability  to  replicate  the  ALERT  forecasting  process  for 
previous  fiscal  years.  ALERT'S  approach  was  advertised  as  superior  to  other  simple 
budget  forecasting  approaches.  However,  the  model  has  never  been  clearly  proven 
superior.  We  need  to  evaluate  the  accuracy  and  validity  of  ALERT  to  forecast  budget 
requirements. 

BACKGROUND 

ALERT  forecasts  budget  requirements  for  three  to  seven  years  out  from  the  present.  As 
ALERT  was  first  used  in  FY84  to  forecast  the  FY87  BP  15  POM,  this  year  was  the  first 
opportunity  we  had  to  validate  the  forecast  against  what  was  actually  spent  (obligated). 
We  need  to  evaluate  the  accuracy  and  validity  of  ALERT  to  forecast  future  budget 
requirements.  Using  available  data,  we  compared  ALERT  to  actual  requirements  figures 
and  to  other  "less  sophisticated"  forecasting  approaches. 

ALERT  uses  CSIS-generated  budget  requirements  for  three  years  beyond  the  present 
fiscal  year,  the  reciprocal  of  age  of  the  fleet,  value  of  the  fleet,  and  indicator  variables 
which  adjust  for  shifts  in  the  data,  against  the  actual  budget  dollars  spent  (or  obligated) 
to  arrive  at  POM  forecast  estimates.  These  estimates  are  generated  by  weapon  system 
and  aggregated  to  develop  the  total  BP  15  POM  forecasts  for  four  to  nine  years  from  the 
current  fiscal  year.  The  forecast  development  process  has  five  basic  steps:  1)  to  update 
and  revise  the  previous  year's  ALERT  data  base,  2)  to  perform  regression  analysis  on  this 
data  base,  3)  to  present  statistically  acceptable  forecasts  for  management  review,  4)  to 
adjust  forecasts,  forecast  equations,  or  forecast  inputs  as  necessary,  and  5)  to  present 
the  ALERT  weapon  system-specific  and  aggregate  forecasts.  ALERT  develops  two  set  of 
forecasts.  The  ALERT  "unscrubbed"  results  reflect  steps  one  through  three  and  are  based 
on  straight  regression.  The  second  set  of  forecasts,  the  "scrubbed"  results,  reflect  the 
necessary  adjustments  required  due  to  updated  data  or  known  future  events  which  might 
impact  the  forecasted  requirement  (for  example  newly  approved  support  for  a  major 
modification). 

OBJECTIVES 

1.  Compare  ALERT  forecasts  for  FY87  (generated  in  1984)  to  actual  BP  15  FY87 
obligations. 

2.  Compare  ALERT  forecasts  to  other  simple  forecasting  methods. 

3.  Identify  possible  improvements  to  ALERT. 
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CHAPTER  2 
ANALYSIS 

We  document  our  analysis  in  three  sections.  In  the  first  section,  we  describe  our 
approach,  including  a  description  of  the  four  forecasting  approaches  we  compared.  The 
four  forecasting  approaches  are  cost  per  flying  hour  (CPFH),  inflation  growth,  unscrubbed 
ALERT,  and  scrubbed  ALERT.  In  the  second  section,  we  compare  the  results  of  the  four 
approaches  to  actual  obligations.  Finally,  we  describe  issues  and  what  actions  we  should 
take  to  improve  future  POM  forecasts. 

APPROACH 

We  compared  four  forecasting  approaches.  The  ALERT  forecast  plus  three  other  possible 
forecasting  approaches  which  could  be  easily  used  to  project  the  BP15  POM.  The  criteria 
for  determining  the  nature  of  these  approaches  were  1)  availability  of  data  to  support 
the  POM  forecasting  approach,  2)  ease  of  use  by  budget  program  managers,  3)  "common 
sense"  selection  of  variables  which  would  be  expected  to  affect  the  BPI5  POM  forecasts. 
The  validation  exercise  involved  comparing  the  following  forecast  methods  to  the  final 
ALERT  results:  the  cost  per  flying  hour  (CPFH)  method  (which  was  used  for  actual  POM 
estimation  before  FY82),  a  forecast  method  based  upon  projected  inflation  rates,  and  the 
"pure"  ALERT  regression  analysis  method  (i.e.,  the  one  without  a  management  scrub). 
Both  the  cost  per  flying  hour  and  the  projected  inflation  rates  have  been  used  in  the 
past  to  forecast  POM  requirements. 

Table  2-1  shows  the  basic  data  used  in  our  analysis. 


BASIC  INPUT  DATA 


Weapon 

System 

A -7 

A-10 

B-52 

C-5 

C-130 

C-135 

C— 141 

E-3 

F-4 

F— 15 

F— 16 

F-111 

F-100 

COMMON 

OTHER 

TOTAL 


FY84 

Obligations 
(IN  $M) 

28.4 

101.1 

136.2 
97.0 

93.1 

54.5 
43.9 
66.0 

181.2 

182.8 
77.0 

395.3 

285.4 
479.9 
173.6 

2395.4 


FY87 

Obligations 
(IN  $M) 

16.6 

46.5 

39.2 

128.5 

69.8 

89.8 
66.7 

47.5 

72.4 

194.4 

146.6 

301.6 

625.8 

330.4 

154.7 

2330.3 


1984 

Flying  Hours 
(IN  000s) 

73.0 

2z2.2 

102.5 

58.1 

378.6 

262.3 
149.5 

30.1 

350.1 

175.7 
186.0 
79.0 

351.2 


2562.6 


1987 

Flying  Hours 
IN  000s) 

73.0 

227.2 

108.1 

66.1 

386.8 

300.2 

292.1 

36.6 

332.8 
216.5 
294.7 

93.2 

433.0 


2860.3 


TABLE  2-1 
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Inflation  Growth 

The  second  approach  used  for  the  validation  of  ALERT  applies  the  1984  inflation  factor 
for  1987  dollars,  which  was  1.128.  The  1984  obligated  dollars  wer^  multiplied  by  1.128 
across  all  weapon  systems  (including  "Common"  and  "Other"  categories)  to  arrive  at  an 
estimate  of  the  1987  POM  submission. 


INFLATION  GROWTH  FORECAST 


FY84 

PROJECTED  FY87 

WEAPON 

OBLIGATIONS 

OBLIGATIONS 

SYSTEM 

(IN  $M) 

(FY84  $  *  1.128) 

A -7 

28.4 

32.0 

A-10 

101.1 

114.0 

B-S2 

136.2 

153.6 

C-5 

97.0 

109.4 

C-I30 

93.1 

105.0 

C-135 

54.5 

61.5 

C-141 

43.9 

49.5 

E-3 

66.0 

74.4 

F-4 

181.2 

204.4 

F— 1 5 

182.8 

206.2 

F-i6 

77.0 

86.9 

F-111 

395.3 

445.9 

F-iOO 

285.4 

321.9 

COMMON 

479.9 

541.3 

OTHER 

173.6 

195.8 

TOTAL 

2395.4 

2702.0 

TABLE  2-3 


ALERT  "Unscrubbed"  Results 

This  lorecasting  approach  is  presented  to  assess  the  impact  of  the  ALERT  "management 
scrub"  upon  the  accuracy  of  the  forecast.  These  values  were  selected  from  the  final 
approved  regression  equations  used  to  baseline  the  ALERT  submission  immediately  before 
the  management  scrub  took  place  [ALERT  project  report,  MMMA  unpublished  draft). 


ALERT  UNSCRUBBED  FORECAST 


WEAPON  SYSTEM 

A -7 

A-10 

B-52 

C-5 

C-130 

C— 135 

C— 141 

E-3 

F-4 

F-15 

F—  1 6 

F-lll 

F-100 

COMMON 

OTHER 

TOTAL 


ALERT 

UNSCRUBBED  RESULTS 

12.7 

48.0 

185.3 

45.2 

90.5 
70.4 
54.0 
45.0 
146.8 

137.5 

418.2 
291.1 

401.4 

322.3 

211.0 

2479.4 


TABLE  2-4 


ALERT  Final  Results 

Before  submission  to  the  Air  Staff,  the  ALERT  forecasts  undergo  the  revision  process. 
This  "management  scrub"  is  necessary  because  there  are  some  problems  with  the  ALERT 
data  base  used  to  develop  the  forecasts.  First,  the  data  base  has  a  limited  number  of 
variables,  t\  >  of  which  (the  age  of  the  fleet,  value  of  the  fleet)  are  computed  by 
SAF/ACCE.  HQ  AFLC/MMMI  does  not  agree  with  the  method  used  to  develop  the  value 
of  the  fleet  estimates,  nor  that  these  variables  accurately  predict  future  requirements. 
They  point  out  the  value  of  the  fleet  does  not  decrease  as  fast  as  the  computed  value 
does,  in  fact  they  claim  it  doesn't  decrease  at  all  for  selected  weapon  systems.  As  a 
result  of  the  current  method  to  compute  the  value  of  the  fleet,  the  requirement  may  be 
underestimated.  Secondly,  the  ALERT  data  base  should  be  expanded  to  include  more 
possible  variables  which  impact  the  BPI5  POM.  One  of  the  weaknesses  of  ALERT  is 
although  the  variables  have  a  relatively  high  statistical  correlation,  they  do  not  have  a 
logical  cause-effect  relationship.  As  a  result,  the  current  ALERT  variables  do  not 
provide  a  complete  "intuitive"  interpretation  of  why  past  relationships  exist.  Thirdly,  the 
"management  scrub"  can  adjust  for  the  effect  of  a  non-programmatic  and  non-statistical 
decisions  upon  the  budget  and  item  requirements.  The  quantitative  effects  of  these 
decisions  cannot  be  accurately  portrayed  in  the  ALERT  model  or  any  other  strictly 
quantitative  model. 
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ALERT  SCRUBBED  FORECAST 


WEAPON  SYSTEM 

A -7 

A-JO 

B-52 

C-5 

C-130 

C— 135 

C— 141 

E-3 

F-4 

F-1S 

F— 16 

F-lll 

F-100 

COMMON 

OTHER 

TOTAL 


ALERT 

SCRUBBED  RESULTS 

12.7 

42.2 

148.2 

74.6 

78.7 
62.0 

47.5 

50.5 
118.9 

126.5 

372.2 
238.7 
393.4 

322.3 
189.1 

2277. 5 


TABLE  2-5 


Comparison  Results 

In  this  section  we  compare  the  FY87  forecasts  for  the  four  forecast  approaches  to  the 
actual  PY87  first  year  (FY87)  close-out  obligated  dollars.  Table  2-6  shows  the  forecasts 
for  all  four  forecasting  methods  and  the  actual  obligations.  Then  Table  2-7  shows  the 
forecast  errors  by  type  of  forecast  approach  for  all  four  methods  above  to  use  FY84 
information  to  project  the  PY87/FY87  BP15  budget. 
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COMPARISON  OF  THE 
FOUR  APPROACHES 
($  Millions) 


Alert 

Alert 

BP15 

Weapon 

Cost  Per 

Inflation 

Unscrubbed 

Final 

FY87/PY87 

System 

Flying  hour 

Growth 

Estimates 

Estimates 

Obligations 

A -7 

28.4 

32.0 

12.7 

12.7 

16.6 

A-10 

103.4 

114.0 

48.0 

42.2 

46.5 

B-52 

143.6 

153.6 

185.3 

148.2 

39.0 

C-5 

110.4 

109.4 

45.2 

74.6 

128.5 

C-130 

95.1 

105.0 

90.5 

78.7 

69.8 

C— 135 

62.4 

61.5 

70.4 

62.0 

89.8 

C— 141 

43.6 

49.5 

54.0 

47.5 

66.7 

E-3 

80.3 

74.5 

45.0 

50.5 

47.5 

F-4 

172.2 

204.4 

146.8 

118.9 

72.4 

F— 1 5 

225.2 

206.2 

137.5 

126.5 

194.4 

F— 16 

122.0 

86.9 

418.2 

372.2 

146.6 

F-Ill 

466.4 

445.9 

291.1 

238.7 

301.6 

F-100 

351.9 

321.9 

401.4 

393.4 

625.8 

COMMON 

* 

541.3 

322.3 

322.3 

330.4 

OTHER  - 

* 

195.8 

211.0 

189.1 

154.7 

TOTAL 

2004.9 

2702.0 

2479.4 

2277.5 

2330.3 

*  Estimates  for  these  categories  included  in  the  major  weapon  system  estimates. 
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FORECAST  ERRORS  FOR  THE 
FOUR  APPROACHES 


Cost  Per 


Weapon 

Flying  Hour 

Inflation  Growth 

Unscrubbed  Alert 

Scrubbed  Alert 

System 

Forecast 

Error 

Forecast  Error 

Forecast  Error 

Forecast  Error 

A -7 

11.8 

15.4 

3.9 

3.9 

A-10 

56.9 

67.5 

1.5 

4.3 

B-52 

104.6 

114.6 

146.3 

109.2 

C-5 

18.1 

19.1 

83.3 

53.9 

C-130 

25.3 

35.2 

20.7 

8.9 

C-135 

27.4 

28.3 

19.4 

27.8 

C— 141 

23.0 

17.2 

12.7 

19.2 

E-3 

32.8 

267.0 

2.5 

3.0 

F-4 

99.8 

132.0 

74.4 

46.5 

F-15 

30.8 

11.8 

56.9 

67.9 

F— 16 

24.6 

59.7 

271.6 

225.6 

F-lll 

164.8 

144.3 

10.5 

62.9 

F-100 

273.9 

303.9 

224.4 

232.4 

COMMON 

330.4 

210.9 

8.1 

8.1 

OTHER  154.7 

TOTAL  BP15 

41.1 

56.3 

34.4 

POM 

325.4 

371.7 

149.1 

52.8 

TOTAL 

ERROR 

1704.5 

1599.8 

1141.6 

960.8 

TABLE  2-7 


Despite  its  weaknesses,  ALERT  is  a  more  accurate  predictor  than  the  other  simple 
forecasting  approaches.  For  12  (9  weapon  systems,  common  and  other,  and  the  total 
BP15  POM)  of  the  16  forecasts,  ALERT  (unscrubbed  or  scrubbed)  provided  more  accurate 
forecasts.  Note  scrubbed  ALERT'S  total  BP15  POM  requirement  was  within  2  percent 
(52.8/2330.3)  of  the  actual  obligation  for  program  year  (PY)  1987  and  fiscal  year  (FY) 
1987.  By  PY87/FY87  we  mean  1987  three-year  program  money  obligated  in  1987.  Also 
note  the  ALERT  scrubbed  values  were  generally  as  accurate  as  the  unscrubbed  values 
except  for  the  total  BP  1 5  POM  which  was  much  more  accurate. 


Although  ALERT  was  more  accurate,  we  wanted  to  see  if  ALERT  provided  statistically 
valid  results. 


To  compare  among  the  limited  number  of  observations,  we  used  a  non-parametric  test 
called  the  chi-square  test: 


Chi-square  = 


Observed  Value  -  Expected  Value 


Observed  Value 


:vi 


>1 


For  this  comparison,  we  considered  the  "observed  value"  to  be  equal  to  the  obligated 
dollars  for  PY87  at  the  close-out  of  FY87.  The  "expected"  or  forecast  value  changed 
depending  upon  which  forecast  approach  was  used.  The  forecasted  values  for  each 
weapon  system  and  the  total  budget  were  considered  as  "expected  values."  We  can 
consider  the  forecasts  to  be  valid  if  the  chi-square  comparison  between  the  forecasted 
POM  and  actual  obligations  is  smaller  than  the  chi-square  standard  (or  critical)  value  of 
7.261.  The  chi-square  test  is  reliable  for  small  samples,  which  makes  it  valuable  in  this 
case,  where  we  have  15  weapon  systems  and  a  total  line  from  which  to  validate  the 
forecasts.  Below  are  the  total  chi-square  values  for  each  of  the  forecast  approaches 
presented  above. 


CHI-SQUARE  TOTALS  FOR 
THE  FOUR  APPROACHES 


COST  PER 
FLYING  HOUR 
APPROACH 
CHI-SQUARE 

1251.11 


INFLATION 

GROWTH 

APPROACH 

CHI-SQUARE 

1126.53 


UNSCRUBBED 

ALERT 

CHI-SQUARE 

1314.47 


SCRUBBED 

ALERT 

CHI-SQUARE 

853.16 


TABLE  2-7 

Table  2-7  indicates  that  none  of  the  forecasting  approaches  are  statistically  close  to  the 
actual  obligated  dollars  for  PY87/FY87.  Table  2-6  shows  that  a  large  part  of  the 
forecast  errors  are  attributable  to  three  weapon  systems:  the  B-52,  the  F-16,  and  the  F- 
100,  all  of  which  are  undergoing  constant  change  and/or  growth.  These  systems  were 
undergoing  considerable  program  volatility,  either  through  expansion  of  the  fleet  (F-16, 
F-100)  or  through  modifications  (B-52).  Accurate  POM  forecasts  for  these  systems  is 
particularly  difficult.  Table  2-8  shows  that  the  chi-square  values  for  the  forecasts  would 
be  significantly  reduced  if  these  three  systems  were  not  considered. 

CHI-SQUARE  TOTALS  WITHOUT  F-16,  F-100,  AND  B-52 


The  statistics  show  there  is  still  room  for  improvement;  however,  a  2  percent  overall 
error  is  really  remarkable  for  a  long-range  forecast.  Of  course  there  is  no  guarantee 
this  accuracy  will  continue  in  the  future;  after  ail  this  is  only  one  data  point.  We 
suggest  this  analysis  be  repeated  yearly  to  continue  to  measure  ALERT'S  validity. 
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FLYING  HOUR 

INFLATION 

UNSCRUBBED 

ALERT  USED 

8 

APPROACH 

APPROACH 

ALERT 

FOR  BP15  POM 

■»> 

CHI-SQUARE 

CHI-SQUARE 

CHI-SQUARE 

CHI-SQUARE 

846.32 

617.68 

182.01 

113.93 

• 

TABLE  2-8 
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ISSUES 


Earlier  in  this  report,  we  addressed  some  of  the  issues  with  ALERT.  In  this  section  we 
highlight  all  the  issues,  discuss  their  impact  and  propose  future  analysis  to  resolve  the 
issues. 

1.  Although  ALERT  uses  regression  and  uses  variables  that  are  highly  correlated  to 
requirements  in  a  statistical  since,  there  doesn't  appear  to  be  any  cause  and  effect 
relationship  between  the  variables.  Optimally,  a  forecasting  model  could  be  used  to 
predict  the  impact  of  policy  decisions  on  requirements.  ALERT  does  not  do  that.  For 
example,  one  can  build  a  statistically  reliable  regression  model  that  shows  the  mmber  of 
drunks  in  a  town  is  directly  related  to  the  number  of  churches.  However,  one  cannot 
conclude  that  if  you  reduce  the  number  of  churches  in  a  town,  the  number  of  drunks  will 
decrease.  We  should  conduct  analysis  to  see  if  ALERT  should  include  variables  like 
actual  flying  hours,  type  of  mission,  and  the  expected  occurrence  of  a  major  modification 
to  predict  cause  and  effect  relationships. 

2.  Further  analysis  is  needed  to  determine  the  sensitivity  of  the  age  of  the  fleet  and 
value  of  the  fleet  on  future  requirements.  The  current  formulas  to  compute  the  value  of 
the  fleet  tends  to  depreciate  the  fleet  value,  and  hence  reduce  future  requirements,  too 
quickly. 

3.  We  need  to  validate  ALERT  annually.  If  there  are  significant  errors  in  the 
forecast,  we  need  to  identify  the  cause  of  the  errors  and  ensure  we  develop  a  "fix"  for 
future  forecasts. 

4.  There  are  a  number  of  other  models  that  forecast  BP15  requirements,  such  as  the 
Peacetime  Operating  Spares  Support  Estimating  Model  (POSSEM),  the  Air  Force  Spares 
Estimating  Model  (AFSEM),  and  MACROSTRAT.  In  addition,  the  RAND  Corporation  is 
beginning  a  study,  "Enhancing  the  Logistics  Requirements  Estimation  Process."  The 
objective  of  RAND's  project  is  to  develop  and  assess  alternative  ways  to  forecast  spares 
requirements.  We  should  compare  the  performance  of  these  models  to  ALERT  to  either 
replace  ALERT  or  identify  possible  improvements  to  ALERT. 

5.  Currently  ALERT  forecasts  requirements  by  weapon  system.  Perhaps  improved 
results  could  result  from  forecasting  requirements  by  commodity  groupings.  For  example, 
electronic  component  prices  tend  to  decline  as  new  technology  is  developed.  Perhaps  a 
different  regression  equation  for  electronic  spares  would  be  more  accurate.  This  would 
require  a  significant  data  base  like  the  one  envisioned  in  the  Requirements  Data  Bank 
Strategic  Data  Base  [I].  Some  analysis  is  needed  to  determine  the  feasibility  and 
accuracy  of  using  commodity  groupings  or  some  other  aggregation  of  the  data.  Then  if 
feasible,  a  data  base  must  be  built. 


CHAPTER  3 


CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS 

1.  In  order  to  validate  ALERT  and  continually  improve  ALERT,  we  should  annually 
compare  the  ALERT  POM  projections  against  actual  obligations  for  the  first  fiscal  year 
of  that  program  year's  budget. 

2.  A  comparison  of  the  1987  ALERT  POM  forecasts  (developed  in  1984)  to  actual  1987 
obligations  shows  ALERT'S  total  BP15  spares  forecast  was  remarkably  accurate  (within  2 
percent). 

3.  Although  not  statistically  accurate,  ALERT  outperformed  other  "less  sophisticated" 
forecasting  models  both  by  weapon  system  and  by  total  BP  15  spares  requirement. 

4.  We  need  to  expand  the  historical  data  base  to  include  other  variables  to  forecast 
the  BP  15  budget  to  reduce  the  forecast  error  and  to  explain  (in  a  cause  and  effect 
manner)  changes  in  requirements. 

5.  We  need  to  examine  other  BP  15  POM  forecasting  models  (either  in-house  or 
contractor)  in  order  to  guarantee  forecast  model  superiority  and  accuracy. 

6.  We  need  to  analyze  the  accuracy  and  sensitivity  of  the  age  of  the  fleet  and  value 
of  the  fleet  variables  on  the  ALERT  model  forecasts. 

RECOMMENDATIONS 

1.  Continue  to  use  ALERT.  (OPR:  HQ  AFLC/MMM) 

2.  Repeat  the  validation  effort  for  FY88  using  ALERT  POM  projections  from  FY85. 
(OPR:  HQ  AFLC/MMMA) 

3.  Continue  to  refine  the  ALERT  model  by  enhancing  the  input  variables  and  the  data 
base.  (OPR:  HQ  AFLC/MMMA  OCR:  HQ  AFLC/MMMI) 

4.  Continue  to  experiment  with  other  BP15  POM  forecasting  approaches  and  validate 
their  results  against  those  of  ALERT.  (OPR:  HQ  AFLC/MMMA  OCR:  HQ  AFLC/MMMI) 
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